Proposing a Future Evaluation Strategy
How to write the future evaluation strategy C10-1 asks for. Read it before class write 1.
The band this serves. C10-1, 9–10: "Proposes an evaluation strategy to be conducted sometime in the future to evaluate the efficiency and effectiveness of the software solution that includes: the time frame for the evaluation to be conducted, the evaluation criteria to be used, the individuals to conduct the evaluation and their responsibilities." All four ingredients, or the band does not open.
flowchart LR
D["Deployment<br/>day 0"] --> U["Real users, real work<br/>3–6 months"]
U --> E["Evaluation window<br/>2–4 weeks"]
E --> R["Findings · decisions about improvements"]
S["System data<br/>logs · usage frequency · crash reports"] --> E
F["User feedback<br/>satisfaction ratings · usability survey"] --> E
The strategy sits on a timeline, which is why "later" never scores: the gap before the window is what gives users time to form a real opinion, and the window itself is what the two data streams feed.
A plan for evaluating your software solution after it's been used for a while by real users in real situations.
Purpose: Conduct a post-launch review of solution usability and performance to ensure ongoing effectiveness.
🎯 Required Components (C10-1)
1. Time Frame - When will the evaluation happen?
- 🔹 When: 3-6 months after software deployment
- 🔹 Why: Users need time to get comfortable with the solution
- 🔹 Duration: 2-4 weeks for data collection and analysis
Example: "Evaluation will be conducted 6 months after deployment (March 2026), taking 3 weeks to complete."
2. Evaluation Criteria - What will be measured?
- 🔹 Start with key questions:
- Is usability still strong?
- Have bugs/errors increased?
- Do all original requirements still work?
- Are users satisfied with performance?
- 🔹 Use your existing criteria from the evaluation matrix
- 🔹 Combine process + outcome measures:
- System data: logs, usage frequency, crash reports
- User feedback: satisfaction ratings, usability surveys
Example: "Evaluate usability, speed of processing, accuracy, and user satisfaction using original criteria plus system performance logs."
⚠️ Use VCAA-named factors only. "Your existing criteria" means the criteria you built in C4 — but double-check them before reusing them here. Worked examples in the wild — including one in the C4 task sheet — list "resource usage" and "response time" as efficiency factors. Neither is a VCAA-named factor. The real list is exactly 3 efficiency factors (Speed of Processing, Cost of Data and File Manipulation, Functionality) and 11 effectiveness factors (Accessibility, Accuracy, Attractiveness, Clarity, Communication of Message, Completeness, Maintainability, Readability, Relevance, Timeliness, Usability) — see Efficiency vs Effectiveness for the full table with descriptions. If your C4 evaluation matrix carried over a non-VCAA factor name, relabel it using the correct factor before you evaluate against it here — a criterion named "resource usage" can usually be reframed as "cost of data and file manipulation" or "speed of processing," depending on what you were actually measuring.
3. Individuals - Who will do the evaluation?
- 🔹 End users (people actually using the software)
- 🔹 Technical evaluator (developer/IT person)
- 🔹 Stakeholder representative (manager/decision maker)
Example: "5 regular users, 1 IT administrator, and 1 department manager will participate."
4. Responsibilities - Who does what?
- 🔹 Users: Provide feedback, complete surveys, participate in interviews
- 🔹 Technical evaluator: Measure performance, analyze usage data
- 🔹 Stakeholder: Review results, make decisions about improvements
📝 Simple Template
Future Evaluation Strategy for [Your Software Name]
When: [Time frame - when and how long]
What: [List 3-5 key criteria to evaluate — VCAA-named efficiency/effectiveness factors, see the warning above]
Who: [List roles and number of people]
How: [Brief description of evaluation methods]
Responsibilities:
- Users: [What they need to do]
- Technical: [What technical person does]
- Manager: [What stakeholder does]
✨ Pro-tip:
Keep it realistic and practical. Think about what would actually happen if someone wanted to check how well your software is working after 6 months of real use.
🚫 Don't Overthink It
This is a proposal, not a detailed research plan. One page is usually enough!
Check Your Understanding
1. A student writes: "We will evaluate the software later using our criteria." Which of the four required ingredients are missing?
Answer
Three, arguably all four. When — "later" is not a time frame. Who — no individuals named. Responsibilities — nothing said about who does what. Even which criteria is only gestured at; the criteria have to be named.
2. Your C4 matrix has a row called "response time". Can you reuse it here as written?
Answer
No. "Response time" is not a VCAA-named factor. Work out what you were actually measuring and relabel it — usually speed of processing. The same goes for "resource usage", which is usually cost of data and file manipulation.
3. Why does the strategy ask for system data and user feedback rather than one of them?
Answer
They answer different halves of the question. System data (logs, crash reports, usage frequency) evidences efficiency and reliability of operation; user feedback evidences effectiveness — whether the solution is still usable, accessible and relevant to the people using it.